Model-free preference-based reinforcement learning

Specifying a numeric reward function for reinforcement learning typically requires a lot of hand-tuning from a human expert. In contrast, preference-based reinforcement learning (PBRL) utilizes only pairwise comparisons between trajectories as a feedback signal, which are often more intuitive to specify. Currently available approaches to PBRL for control problems with continuous state/action spaces require a known or estimated model, which is often not available and hard to learn. In this paper, we integrate preference-based estimation of the reward function into a model-free reinforcement learning (RL) algorithm, resulting in a model-free PBRL algorithm. Our new algorithm is based on Relative Entropy Policy Search (REPS), enabling us to utilize stochastic policies and to directly control the greediness of the policy update. REPS decreases exploration of the policy slowly by limiting the relative entropy of the policy update, which ensures that the algorithm is provided with a versatile set of trajectories, and consequently with informative preferences. The preference-based estimation is computed using a sample-based Bayesian method, which can also estimate the uncertainty of the utility. Additionally, we also compare to a linear solvable approximation, based on inverse RL. We show that both approaches perform favourably to the current state-of-the-art. The overall result is an algorithm that can learn non-parametric continuous action policies from a small number of preferences

Repeat Ablation for Atrial Fibrillation Recurrence Post Cryoballoon or Radiofrequency Ablation in the FIRE and ICE Trial

Background: The FIRE AND ICE trial assessed efficacy and safety of pulmonary vein (PV) isolation using cryoballoon versus radiofrequency current (RFC) ablation in patients with drug refractory, symptomatic, paroxysmal atrial fibrillation (AF). The purpose of the current study was to assess index lesion durability as well as reablation strategy and outcomes in trial patients undergoing a reablation procedure. Methods: Patients with reablation procedures during FIRE AND ICE were retrospectively consented and enrolled at 13 trial centers. The first reablation for each patient was included in the analysis. Documented arrhythmias before reablation, number and location of reconnected PVs, lesions created during reablations, procedural characteristics, and acute as well as long-term outcomes were assessed. Results: Eighty-nine (36 cryoballoon and 53 RFC) patients were included in this study. Paroxysmal atrial fibrillation was the predominant recurrent arrhythmia (69%) before reablation. Reablations occurred at a median of 173 and 182 days (P=0.54) in the cryoballoon and RFC cohorts, respectively. The number of reconnected PVs was significantly higher in the RFC than the cryoballoon group (2.1\ub11.4 versus 1.4\ub11.1; P=0.010), which was driven by significantly more reconnected left superior PVs and markedly more reconnected right superior PVs. The number of (predominantly RFC) lesions applied during reablation was significantly greater in patients originally treated with RFC (3.3\ub11.3 versus 2.5\ub11.5; P=0.015) with no difference in overall acute success (P=0.70). After reablation, no differences in procedure-related rehospitalization or antiarrhythmic drug utilization were observed between cohorts. Conclusions: At reablation, patients originally treated with the cryoballoon had significantly fewer reconnected PVs, which may reflect RFC catheter instability in certain left atrial regions, and thus required fewer lesions for reablation success. Repeat ablations were predominantly performed with RFC and resulted in similar acute success, duration of hospitalization, and antiarrhythmic drug prescription between the study cohorts. Clinical Trial Registration: URL: https://www.clinicaltrials.gov. Unique identifier: NCT03314753

Machine Learning in Automated Text Categorization

The automated categorization (or classification) of texts into predefined categories has witnessed a booming interest in the last ten years, due to the increased availability of documents in digital form and the ensuing need to organize them. In the research community the dominant approach to this problem is based on machine learning techniques: a general inductive process automatically builds a classifier by learning, from a set of preclassified documents, the characteristics of the categories. The advantages of this approach over the knowledge engineering approach (consisting in the manual definition of a classifier by domain experts) are a very good effectiveness, considerable savings in terms of expert manpower, and straightforward portability to different domains. This survey discusses the main approaches to text categorization that fall within the machine learning paradigm. We will discuss in detail issues pertaining to three different problems, namely document representation, classifier construction, and classifier evaluation.Comment: Accepted for publication on ACM Computing Survey

Finding microRNA regulatory modules in human genome using rule induction

Background: MicroRNAs (miRNAs) are a class of small non-coding RNA molecules (20-24 nt), which are believed to participate in repression of gene expression. They play important roles in several biological processes (e.g. cell death and cell growth). Both experimental and computational approaches have been used to determine the function of miRNAs in cellular processes. Most efforts have concentrated on identification of miRNAs and their target genes. However, understanding the regulatory mechanism of miRNAs in the gene regulatory network is also essential to the discovery of functions of miRNAs in complex cellular systems. To understand the regulatory mechanism of miRNAs in complex cellular systems, we need to identify the functional modules involved in complex interactions between miRNAs and their target genes. Results: We propose a rule-based learning method to identify groups of miRNAs and target genes that are believed to participate cooperatively in the post-transcriptional gene regulation, so-called miRNA regulatory modules (MRMs). Applying our method to human genes and miRNAs, we found 79 MRMs. The MRMs are produced from multiple information sources, including miRNA-target binding information, gene expression and miRNA expression profiles. Analysis of two first MRMs shows that these MRMs consist of highly-related miRNAs and their target genes with respect to biological processes. Conclusion: The MRMs found by our method have high correlation in expression patterns of miRNAs as well as mRNAs. The mRNAs included in the same module shared similar biological functions, indicating the ability of our method to detect functionality-related genes. Moreover, review of the literature reveals that miRNAs in a module are involved in several types of human cancer

withdrawn 2017 hrs ehra ecas aphrs solaece expert consensus statement on catheter and surgical ablation of atrial fibrillation

n/

Automatically Evolving Rule Induction Algorithms

Research in the rule induction algorithm field produced many algorithms in the last 30 years. However, these algorithms are usually obtained from a few basic rule induction algorithms that have been often changed to produce better ones. Having these basic algorithms and their components in mind, this work proposes the use of Grammar-based Genetic Programming (GGP) to automatically evolve rule induction algorithms. The proposed GGP is evaluated in extensive computational experiments involving 11 data sets. Overall, the results show that effective rule induction algorithms can be automatically generated using GGP. The automatically evolved rule induction algorithms were shown to be competitive with well-known manually designed ones. The proposed approach of automatically evolving rule induction algorithms can be considered a pioneering one, opening a new kind of research area